This invention relates to techniques for inspecting and classifying images, and more particularly relates to automated classification techniques that enable real time classification of images.
In many operations such as manufacturing processes, it is desirable to inspect images of articles as the articles move through a sequence of operations to ascertain if the articles meet quality criteria or contain unwanted defects that require special processing. For example, during a process for continuous casting of steel, it is preferable to inspect a continuous steel ribbon or cut strips of the ribbon in real time as they pass between heating and cooling chambers, whereby ribbon sections or strips found to meet quality criteria can be immediately directed to a next process, e.g., rolling, without loss of time and without a change in steel temperature. Similarly, processes for producing sheets of paper, plastic, or other such "web" materials gain benefits in efficiency from real time quality inspection of the materials during the manufacturing process; and in general, a wide range of manufacturing and processing operations optimally include provisions for real time inspection of articles during their manufacture.
Inspection for minimum manufacturing quality criteria is typically preferably based on classification of aspects of articles such as, e.g., defects of the articles, as a function of, e.g., defect type and severity. In such a defect classification technique, in the case of a web material, for example, defects of the material are identified in images of the material and measurements, i.e., features, of the defects are extracted from the images. The defects are then typically classified into categories, or classes, such as "scratch," "oil spot," "dirt," "roll mark," or other named defects, as well as subcategories, or subclasses, such as "small scratch," or "large scratch."
Conventional systems for real time inspection and classification, used, e.g., for quality control inspection, typically are implemented as a rule-based, look-up-table classifier configuration in which feature measurements such as defect feature measurements are compared with pre-specified table rules for making classification decisions. For example, Fant et al., in U.S. Pat. No. 4,519,041, describe a rule-based defect classification system employing hierarchical rule-based classification logic in combination with syntactic/semantic classification logic to make classification decisions. Rule-based systems like this generally employ classifier tables that include a set of logical or numerical tests for making the comparison between measured feature values and predefined test feature values or test feature value thresholds.
Generally, to implement a rule-based classification system, a system user must manually determine, through observation, which features are important for defining each of the classification categories of interest. In the case of a material manufacturing process, this can be a complex procedure that requires a skilled engineer or technician who has familiarity both with, e.g., defects of interest to be classified for a given material, as well as a good working knowledge of statistics and mathematics. The complexity of this procedure is compounded by the fact that frequently, a given manufacturing plant has a unique language and folklore for describing and classifying defects, because the nature of defects inherently depends heavily on characteristics of a particular material being manufactured, the process sequence being followed, and the particular equipment being used. Further, machine operators of a given plant frequently prefer definition of more than a general defect class; for example, they may prefer to have a severity index unique to a given operation assigned to classified defects so that appropriate plant personnel can be alerted immediately when a "killer" defect appears and so that they are not bothered by "nuisance" defects that have no real adverse affect on the manufactured product.
But even within a given manufacturing plant or process sequence, a wide variety of materials and types of material defects may be critically important, and a single rule-based classification system is typically responsible for all such materials and characteristic defects. This requires that the classification system be configured with a capability to generate a large number of defined defect features such that optimum classification across the variety of materials and defect classes is ensured. As a result, configuring a rule-based classification system for a given material can be complex in that the system operator typically must select those features that are preferred for a given material inspection process from a substantial amount of defect feature data generated by the system. Thus, while the structure of a rule-based classifier table is generally considered easy to implement and use, the development of an optimum table for a given classification operation is in practice often an unachievable goal.
Indeed, experience in the field of real time material defect classification has shown that very few manufacturing process users of rule-based classification systems have the ability to properly define an optimum, complete set of rule-based table rules for a given material manufacturing operation, and thus, rule-based classification systems generally do not in practice achieve desired levels of classification accuracy. Further, the binary nature of rule-based decisions as implemented in a rule-based classifier often give rise to poor classification repeatability because feature measurement data is subject to quantization errors. The binary nature of rule-based decision making is not flexible because test feature values in a rule-based table are fixed, and thus a rule-based test of a feature results in an a priori classification decision even if a measured feature value and a test feature value differ by as little as, e.g., 1 percent or less. In addition, the performance of rule-based classification systems depends strongly on which test feature values were selected during development of the rule-based table.
Trainable pattern classifiers have been developed which overcome some of the problems of rule-based classifiers mentioned above, in particular by providing the ability to achieve relatively more reliable classification without the need for highly skilled personnel to develop an optimized classification table. This is enabled by the characteristic of trainable pattern classifiers to have an ability to "learn" from a set of training samples, which are articles of known classes, how to make classification decisions based on test features of articles under inspection.
Trainable pattern classifiers are, however, limited in their initial classification capabilities in that until a trainable classifier has been presented with a large number of training samples that cover the full range of possible features for a set of classes of interest, a trainable classifier cannot make effective classification decisions. Thus, the optimum training sample set for each intended class must not only be large, but also well-representative of the extent of the class; presentation to a trainable classifier of even thousands of training samples will not be effective if the samples are similar--a wide range of samples for each class must instead be provided. This is in part because the use of relatively non-unique samples, i.e., samples having features that do not well-distinguish between the boundaries of different classes, can seriously degrade classification results.
Trainable classifiers generally are also distinguished over rule-based classifiers in that trainable classifiers can be configured to regard selected possible classes, e.g., defect classes, as being more critical than others. For example, a trainable classifier can generally be configured such that in selecting between a first class assignment or second class assignment for a given defect the classifier will give priority to, say, the first class, based on, e.g., a loss value prespecified for each class and factored into the classification assignment result. While such selectivity is generally quite important for a manufacturing process, the task of implementing its configuration can be quite complicated, based on particular prespecified dependencies of the loss value, and this complexity often precludes its implementation.
Even without the implementation of this particular trainable classifier configuration, a great deal of time is typically required to identify and gather an optimum set of training samples characteristic of a given manufacturing process and materials to be inspected in general. When such a training set has been accumulated, the training process during which the samples are "taught" to the system can require many hours or even days. This heavy training requirement and the complicated system configuration requirements are not generally reasonable in the fast-paced environment of a manufacturing process.